Monitor device

ABSTRACT

A monitor ECU  10  performs a support control based on a camera image photographed through a protection window by a camera  21 . The ECU determines that a protection window state is an entire dirt state, when a first area index value calculated based on edge strength of pixels in a first area including a center of the camera image is smaller than a threshold dirt value. The ECU determines that the entire dirt state ends, when at least one of a first condition or a second condition is established. The first condition is established when the first area index value is equal to or greater than a first threshold end value. The second condition is established when a second area index value calculated based on the edge strength in a second area except the first area is equal to or greater than a second threshold end value.

BACKGROUND Technical Field

The present invention relates to a monitor device for performing asupport control to support driving of an own vehicle based on a cameraimage photographed by a camera for photographing an area around the ownvehicle through a protection window.

Related Art

Hitherto a device has been known, which detects a lane marker, othervehicle, a pedestrian, a traffic sign, a parking area, and the like,based on an camera image photographed by a camera so as to supportdriving of an own vehicle. When dirt (such as a water droplet, fogging,a water droplet trace, mud, and the like) adheres to a lens of thecamera, the device may inappropriately perform the support control.

For example, as proposed in Japanese Patent Application Laid-open No.2015-95886, a device has been known, which detects the above dirt basedon edge strength of each of pixels included in the camera image. Whenthe proposed device detects the dirt, the device stops the supportcontrol.

SUMMARY

Generally, the edge strength of pixels located in a center area in thevicinity of the center of the camera image tends to be calculatedaccurately, however the edge strength of pixels located in a peripheralarea of the camera image tends to be calculated to be a value smallervalue than a true value. As a view angle of the lens of the camera iswider, the above tendency becomes stronger. However, the device detectsthe dirt without taking such a property of the camera image intoconsideration. Therefore, the device may sometimes be unable toaccurately determine whether the dirt has adhered (or is present) andwhether the adhered dirt has already been removed (or is no longerpresent).

The present invention has been made to solve the problem describedabove. The present invention has an object to provide a monitor devicethat can more accurately determine whether or not the dirt is present.

A monitor device (hereinafter, referred to as a “present inventiondevice”) according to the present invention comprises:

a control unit (10) configured to perform a support control to supportdriving of an own vehicle based on a camera image which includes pixelsand which is acquired by a camera (21) for photographing an area aroundthe own vehicle through a protection window (22) which is exposed tooutside of the own vehicle; and

a determination unit (10) configured to:

-   -   determine whether or not a protection window state is an entire        dirt state where dirt has adhered to an entire surface of the        protection window, based on edge strength (ES) of the pixels        (Step 740); and    -   determine whether or not the entire dirt state ends based on the        edge strength, when it has been determined that the protection        window state has been the entire dirt state (Step 940, Step        975).

The control unit is configured not to perform the support control, whenit has been determined that the protection window state has been theentire dirt state (“No” at Step 630).

The determination unit is configured to:

-   -   determine that the protection window state is the entire dirt        state, when a first area index value (a center edge area number        CEN) calculated based on the edge strength of pixels included in        a first area (a center area CA) is smaller than a threshold dirt        value (a threshold area number CEN1th) (“Yes” at Step 740), the        first area encompassing a center of the camera image; and    -   determine that the entire dirt state ends (Step 945), when at        least one of a first condition or a second condition is        established,    -   the first condition being a condition that the first area index        value is equal to or greater than a first threshold end value (a        threshold area number CEN1th), and    -   the second condition being a condition that a second area index        value (an outer edge area number OEN) calculated based on the        edge strength of pixels included in a second area (an outer area        OA) other than (except) the first area in the camera image is        equal to or greater than a second threshold end value (a        threshold area number OEN1th).

The edge strength in the first area encompassing/including the center ofthe camera image tends to be calculated more accurately than the edgestrength in the second area other than (except) the first area in thecamera image. The present invention device determines whether or not theprotection window state is the entire dirt state based on the “firstarea index value calculated based on the edge strength of each of pixelsincluded in the first area where the edge strength tends to becalculated more accurately”. In other words, the present inventiondevice performs this determination without using the “second area indexvalue calculated based on the edge strength of each of pixels includedin the second area where the edge strength tends to be calculated lessaccurately”. Therefore, the present invention device can more accuratelydetermine whether or not the protection window state is the entire dirtstate.

Further, the present invention device determine that the entire dirtstate ends, when at least one of the first condition or the secondcondition is established. The second condition is established, when the“second area index value calculated based on the edge strength of pixelseach of which is included in the second area where the edge strengthtends to be calculated less accurately” is equal to or greater than thesecond threshold end value. Therefore, even when the camera 21photographs a scene from which no edge is extracted/detected in thefirst area after the entire dirt has been removed, the present inventioncan more accurately determine that the entire dirt state ends.

According to an embodiment of the present invention device,

the determination unit is configured to:

-   -   calculate (Step 935), as the first area index value (CEN), a        value correlating with the number of pixels included in the        first area, each of the pixels in the first area having the edge        strength which is equal to or greater than a first threshold        strength (ES1th); and    -   calculate (Step 970), as the second area index value (OEN), a        value correlating with the number of pixels included in the        second area, each of the pixels in the second area having the        edge strength which is equal to or greater than a second        threshold strength (ES1th).

The value correlating with the number of “pixels included in the firstarea, each of the pixels included in the first area having the edgestrength which is in the first area having the edge strength which isequal to or greater than the first threshold strength” is calculated asthe first area index value. The value correlating with the number of“pixels included in the second area, each of the pixels in the secondarea having the edge strength which is equal to or greater than a secondthreshold strength” is calculated as the second area index value. Theembodiment, which utilizes those index values, can determine that theprotection window state is the entire dirt state more accurately, andcan determine that the entire dirt state ends more accurately.

According to an embodiment of the present invention device, the firstarea is divided into a plurality of individual areas (AR), and

the determination unit is configured to:

-   -   calculate (Step 735), as the first area index value, a first        edge area number (CEN) indicative of the number of individual        areas, each of the individual areas satisfying a condition that        the number of pixels (an edge pixel number EN) included in each        of the individual areas is equal to or greater than a first        threshold pixel number (EN1th), each of the pixels having the        edge strength equal to or greater than the first threshold        strength; and    -   determine that the protection window state is the entire dirt        state (Step 745), when the first edge area number is smaller        than a first threshold area number serving as the threshold dirt        value (“Yes” at Step 740).

“The first edge area number indicative of the number of individualareas, each of the individual areas satisfying a condition that thenumber of pixels included in each of the individual areas is equal to orgreater than a first threshold pixel number, each of the pixels havingthe edge strength equal to or greater than the first threshold strength”is serving as the first area index value. The embodiment can determinethat the protection window state is the entire dirt state moreaccurately, and can determine that the entire dirt state ends moreaccurately.

According to an embodiment of the present invention device, the secondarea is divided into a plurality of individual areas (AR), and

the determination unit is configured to:

-   -   calculate (Step 970), as the second area index value, a second        edge area number (OEN) indicative of the number of individual        areas, each of the individual areas satisfying a condition that        the number of pixels (an edge pixel number EN) included in each        of the individual areas is equal to or greater than a second        threshold pixel number, each of the pixels having the edge        strength equal to or greater than the second threshold strength        (EN1th); and    -   determine that the second condition is established, when the        second edge area number is equal to or greater than a second        threshold area number (OEN1th) serving as the second threshold        end value (“No” at Step 975).

“The second edge area number indicative of the number of individualareas, each of the individual areas satisfying a condition that thenumber of pixels included in each of the individual areas is equal to orgreater than a second threshold pixel number, each of the pixels havingthe edge strength equal to or greater than the second thresholdstrength” is served as the second area index value. The embodiment candetermine that the entire dirt state ends more accurately.

According to an embodiment of the present invention device, the firstarea and the second area are divided into a plurality of individualareas (AR), and

the determination unit is configured to:

-   -   determine that the protection window state is a partial dirt        state where the dirt has adhered to a part of the surface of the        protection window corresponding to an unchanged area (UCA) which        is the individual area where the number of the pixels (UCPN)        satisfying a third condition is equal to or greater than a        threshold pixel number (UCPN1th), the third condition being a        condition that a change amount in a pixel value of each of the        pixels included in the individual area in a predetermined time        period is equal to or smaller than a threshold amount (Step        855), when the camera image includes the unchanged area (“Yes”        at Step 850), and

the control unit is configured not to perform the support control, whenit has been determined that the protection window state has been thepartial dirt state (“No” at Step 630).

The embodiment detects the dirt corresponding to “the unchanged area(UCA) which is the individual area where the number of the pixels whosepixel values do not change (remain unchanged) substantially for/over thepredetermined time period” as partial dirt. The embodiment does notperform the support control based on the inaccurate camera image whichis photographed through the protection window to which the partial dirthas adhered.

According to an embodiment of the present invention device,

in the event of the determination unit determining that the protectionwindow state has been the partial dirt state, the determination unit isconfigured to determine that the partial dirt state corresponding to theunchanged area ends (Step 965), when the number of the pixels (EN)becomes equal to or greater than a third strength threshold, each of thepixels being included in the unchanged area and having the edge strengthequal to or greater than a third strength threshold (EN1th) (“Yes” atStep 960).

The embodiment determines whether or not the partial dirt state endswithout using data in the areas other than/except the unchanged area.Therefore, the embodiment can determine that the partial dirt state endsmore accurately.

According to an embodiment of the present invention device,

the determination unit is configured to determine that the protectionwindow state is the entire dirt state, when the own vehicle starts to beoperated for running.

Snow, frost, water droplets, or the like may adhere to the protectionwindow in a period from a time point at which the own vehicle is parkedto a time point at which the own vehicle starts to be operated forrunning (driving). Thus, it is reasonable to assume that the dirt hasadhered to the entire surface of the protection window in that period.In view of the above, the embodiment determines that the protectionwindow state is the entire dirt state, when the own vehicle starts to beoperated for running (or be ready for running). The embodiment canprohibit itself from performing the pre-collision control based on basedon the inaccurate camera image which is photographed through theprotection window to which the entire dirt is likely to has adhered whenthe own vehicle starts to be operated for running.

In the above description, in order to facilitate the understanding ofthe invention, reference symbols used in embodiment of the presentinvention are enclosed in parentheses and are assigned to each of theconstituent features of the invention corresponding to the embodiment.However, each of the constituent features of the invention is notlimited to the embodiment as defined by the reference symbols.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic system configuration diagram of a monitor device(this monitor device) according to an embodiment.

FIG. 2 is a diagram illustrating a position at which a camera isarranged, positions at which clearance sonars are arranged, and apredicted travel path.

FIG. 3 is a diagram illustrating a center area and an outside area in acamera image.

FIG. 4 is a diagram illustrating a camera image photographed in a statewhere dirt is adhered to a protection window.

FIG. 5 is a diagram illustrating a camera image of which no edge isextracted in the center area.

FIG. 6 is a flowchart illustrating a routine which is executed by a CPUof a monitor ECU.

FIG. 7 is a flowchart illustrating another routine which is executed bythe CPU of the monitor ECU illustrated in FIG. 1.

FIG. 8 is a flowchart illustrating another routine which is executed bythe CPU of the monitor ECU illustrated in FIG. 1.

FIG. 9 is a flowchart illustrating another routine which is executed bythe CPU of the monitor ECU illustrated in FIG. 1.

DETAIL DESCRIPTION

A monitor device (hereinafter referred to as a “this monitor device”)according to an embodiment of the present invention will next bedescribed with reference to the accompanying drawings. A vehicle inwhich the control device is installed is referred to as an “own vehicleSV”, when this vehicle needs to be distinguished from other vehicles.

This monitor device performs a support control to support driving of theown vehicle SV based on a camera image which a camera 21 included in acamera system 20 shown in FIG. 1 photographs through a protection window22. This monitor device determines whether or not dirt has adhered tothe protection window 22. When this monitor device determines that thedirt has adhered to the protection window 22, the monitor deviceprohibits itself from performing (or does not perform) the supportcontrol based on the camera image which the camera 21 photographsthrough the protection window 22 to which the dirt has adhered.

As shown in FIG. 1, this monitor device comprises a monitor ECU 10. Themonitor ECU 10 comprises a microcomputer including a CPU 11, a ROM 12, aRAM 13, and the like. It should be noted that an ECU is an abbreviationof an “Electronic Control Unit” which includes a microcomputer as a mainpart. The microcomputer includes a CPU, and memories (for example, aROM, a RAM, and the like). The CPU achieves various functions byexecuting instructions (program, routine) stored in the ROM.

This monitor device comprises a camera system 20, clearance sonars 24Athrough 24D, a shift position sensor 25, a vehicle state sensor 26, adisplay unit 30, a speaker 31, a brake ECU 32, a brake sensor 33, abrake actuator 34, an engine ECU 35, and an engine actuator 36. Themonitor ECU 10 is connected to the camera system 20, the clearancesonars 24A through 24D, the shift position sensor 25, the vehicle statesensor 26, the display unit 30, the speaker 31, the brake ECU 32, andthe engine ECU 35. The clearance sonars 24A through 24D are collectivelyreferred to as “clearance sonars 24”.

The camera system 20 includes the camera 21 and an image processingdevice 23. As shown in FIG. 2, the camera 21 is arranged at the centerin a width direction in a rear end of the own vehicle SV. Referring backto FIG. 1, the camera 21 is attached to an outer side of the own vehicleSV. The camera 21 has the protection window 22 which is a transparentplate. The protection window 22 protects a lens of the camera 21 from awater droplet, mud, dust, and the like. The camera 21 photographs ascene of a backward area of the own vehicle SV through the protectionwindow 22. It should be noted that an angle of view of the camera 21 iswide, i.e. about 180 deg. The camera 21 photographs the scene totransmit an image (a camera image or camera image data) photographed bythe camera 21 to the image processing device 23, every time apredetermined time period elapses.

The image processing device 23 selects/extracts an object whose type iscoincident with one of predetermined types (a pedestrian, a vehicle, amotorcycle, a bicycle, and the like) from the camera image photographedby the camera 21. More specifically, the image processing device 23stores an image feature amount of the object of each of thepredetermined types as a matching pattern in advance. The imageprocessing device 23 divides the camera image into local areas, each ofwhich has a predetermined size, so as to calculate the image featureamount of each of the local areas. The image processing device 23compares the calculated image feature amount with each image featureamount which is stored as the matching pattern so as to select/extractthe object from the camera image. The image processing device 23transmits, to the monitor ECU 10, the camera image data which includesthe camera image, type data expressing the type of theselected/extracted object, and position data expressing a position ofthe selected/extracted object in the camera image, every time apredetermined time period elapses.

Each of the clearance sonars 24 detects a location of a 3D object(object) which is located in the backward area of the own vehicle SV anda relative velocity of the object in relation to the own vehicle SVusing an ultrasonic wave. More specifically, each of the clearancesonars 24 emits (transmits) the ultrasonic wave. When the object ispresent within an emission range of the ultrasonic wave, the objectreflects the ultrasonic wave. When the clearance sonar 24 receives thereflected ultrasonic wave, the clearance sonar 24 calculates a distancebetween the own vehicle SV and the object based on a time period from atime point of the emission of the ultrasonic wave to a time point of thereceipt of the ultrasonic wave, and a direction of the object inrelation to the own vehicle SV based on the direction of the reflectedultrasonic wave. The location of the object is specified based on “thedistance between the own vehicle SV and the object” and the direction ofthe object in relation to the own vehicle SV. Further, the clearancesonar 24 calculates the relative velocity of the object based onfrequency change (by the Doppler effect) of the reflected ultrasonicwave. The clearance sonars 24 transmit object information including thelocation of the object and the relative velocity of the object to themonitor ECU 10.

As shown in FIG. 2, the clearance sonars 24A through 24D are arranged atthe rear end of the own vehicle SV so as to be spaced from each other bya predetermined distance in the width direction of the own vehicle SV.The clearance sonar 24A is arranged at a right end of the rear end ofthe own vehicle SV so as to detect an object which is present in an areaDRA which expands around a right corner in the backward area of the ownvehicle SV. The clearance sonar 24B is arranged at a right side of therear end of the own vehicle SV so as to detect an object which ispresent in an area DRB which expands toward a right backward of the ownvehicle SV. The clearance sonar 24C is arranged at a left side of therear end of the own vehicle SV so as to detect an object which ispresent in an area DRC which expands toward a left backward of the ownvehicle SV. The clearance sonar 24D is arranged at a left end of therear end of the own vehicle SV so as to detect an object which ispresent in an area DRD which expands around a left corner in thebackward area of the own vehicle SV.

Referring back to FIG. 1, the shift position sensor 25 detects aposition (hereinafter referred to as a “shift position”) of a shiftlever to be operated by the driver so as to transmit a signal indicativeof the detected shift position. The shift position includes a parkingrange “P”, a drive range “D”, a reverse range “R”, a neutral range “N”,and the like. The monitor ECU 10 acquires (detects) the shift positionfrom the shift position sensor 25, every time a predetermined timeperiod elapses.

The vehicle state sensor 26 includes a vehicle velocity sensor fordetecting a velocity (a vehicle velocity) Vs of the own vehicle SV, anacceleration sensor for detecting an acceleration As in a front-reardirection in a horizontal plane of the own vehicle SV and anacceleration As in a left-right direction in the horizontal plane of theown vehicle SV, a yaw rate sensor for detecting a yaw rate Yr of the ownvehicle SV, a steering angle sensor for a steering angle θ of steeredwheels, and the like. The monitor ECU 10 acquires (detects) vehiclestate information which includes the vehicle velocity Vs, theaccelerations As, the yaw rate Yr, and the steering angle θ, every timea predetermined period elapses.

When the shift position acquired from the shift position sensor 25 is“R”, in other words, when the own vehicle SV runs backwards, the monitorECU 10 fuses the camera image information with the object information soas to specify the location of the object which is present in thebackward area of the own vehicle SV. When the object with a highprobability of colliding with the own vehicle SV is present in thebackward area of the own vehicle SV, the monitor ECU 10 alerts/warns thedriver of the presence of the object, using the display unit 30 and thespeaker 31. If the object's probability of colliding with the ownvehicle SV becomes higher than the object's probability at the timepoint at which the monitor ECU 10 alerted/warmed the driver, the monitorECU 10 controls the brake ECU 32 and the engine ECU 35 such that thevehicle velocity Vs of the own vehicle SV is decreased so as to stop theown vehicle SV before the own vehicle SV collides with the object.

The display unit 30 is a “Head Up Display” (hereinafter referred to as a“HUD”) which receives display information from each of the ECUs in theown vehicle SV and a navigation device, and displays the receiveddisplay information on a partial area (a display area) of a front glassof the own vehicle SV. The display unit 30 displays an alert screenwhich has the driver pay attention to the object (obstacle) which hasthe high probability of colliding with the own vehicle SV and which ispresent in the backward area of the own vehicle SV. When the displayunit 30 receives a display instruction signal which is a displayinstruction to display the alert screen from the monitor ECU 10, thedisplay unit 30 displays the alert screen. In some embodiments, thedisplay unit 30 is a liquid-crystal display.

When the speaker 31 receives, from the monitor ECU 10, an outputinstruction signal to output/generate alert sound, the speaker 31outputs/generates the alert sound in order to have the driver payattention to the obstacle in response to the received output instructionsignal.

The brake ECU 32 is connected to “the vehicle velocity sensor fordetecting the vehicle velocity Vs”. The vehicle velocity sensor isincluded in the vehicle state sensor 26. The brake ECU 32 is connectedto brake sensors 33. The brake ECU 32 receives detection signals fromthose sensors. The brake sensors 33 detects parameters which the brakeECU 32 uses when the brake ECU 32 controls a brake device (not shown)installed in the own vehicle SV. The brake sensors 33 includes a sensorfor detecting a brake pedal operating amount (depressing amount), or thelike.

The brake ECU 32 is connected to a brake actuator 34. The brake actuator34 is a hydraulic control actuator. The brake actuator 34 is provided inan unillustrated hydraulic circuit between an “unillustrated mastercylinder which pressurizes working oil by using a depressing forceapplied to the brake pedal” and “unillustrated friction brake mechanismsincluding well-known wheel cylinders”. Each of the wheel cylinders isprovided in each of wheels. The brake actuator 34 adjusts oil pressureapplied to the wheel cylinder. The brake ECU 32 drives the brakeactuator 34 so as to generate braking force (frictional braking force)at each of the wheels to thereby adjust the acceleration (a negativeacceleration, i.e. a deceleration) of the own vehicle SV.

When the brake ECU 32 receives a brake instruction signal from themonitor ECU 10, the brake ECU 32 controls the brake actuator 34 suchthat an actual acceleration As of the own vehicle SV matches a targetdeceleration TG included in the brake instruction signal, to therebydecrease the vehicle velocity Vs through braking. It should be notedthat the monitor ECU 10 acquires the acceleration As of the own vehicleSV from the acceleration sensor included in the vehicle state sensor 26.

The engine ECU 35 is connected to the engine actuator 36. The engineactuator 36 changes an operational state of an engine (not shown) whichis a driving source of the own vehicle SV. The engine actuator 36includes at least one throttle valve actuator for changing opening of athrottle valve. The engine ECU 35 drives the engine actuator 36 so as tochange torque generated by the engine. In this manner, the engine ECU 35can change driving force of the own vehicle SV. When the monitor ECU 10transmits the brake instruction signal to the brake ECU 32, the monitorECU 10 transmits a torque decreasing instruction signal to the engineECU 35. When the engine ECU 35 receives the torque decreasinginstruction signal, the engine ECU 35 drives the engine actuator 36 (inactuality, the engine ECU 35 drives the throttle valve actuator so as tochange the opening of the throttle valve to the minimum opening) tothereby change the torque of the engine to the minimum torque.

<Outline of Operation>

An operation of this monitor device will next be described. As describedabove, this monitor device detects the object(s) which is present in thebackward area of the own vehicle SV based on the camera image taken bythe camera 21 and the detection results of the clearance sonars 24. Thismonitor device selects/extracts the obstacle(s) with the highprobability of colliding with the own vehicle SV among the detectedobjects. This monitor device calculates a collision time period (time tocollision) TTC indicative of a time period which it takes the obstacleto collide with own vehicle SV or to reach the closest point to the ownvehicle SV. A process for selecting/extracting the obstacle and aprocess for calculating the time to collision TTC will be describedlater.

When the time to collision TTC is equal to or shorter than a timethreshold T1th for an alert control, this monitor device transmits theabove display instruction signal to the display unit 30 and the aboveoutput instruction signal to the speaker 31 so as to perform the alertcontrol for alerting the driver of the presence of the obstacle. Thealert control is one of support controls which support the driving bythe driver.

When the time to collision TTC is equal to or shorter than a timethreshold T2th for a collision prevention control, this monitor devicecalculates the target deceleration TG which is required to stop the ownvehicle SV before the own vehicle SV collides with the obstacle. Itshould be noted that the time threshold T2th is set to a value smallerthan the above time threshold T1th. This monitor device transmits theabove brake instruction signal including the target deceleration TG tothe brake ECU 32 and the above torque decreasing instruction signal tothe engine ECU 35 so as to perform a brake control. The brake control isone of the support controls.

As described above, this monitor device detects the object based on theabove camera image and the detection results of the clearance sonars 24.Therefore, when dirt has adhered to (or is present on) the protectionwindow 22 of the camera 21, this monitor device becomes unlikely to beable to detect the object accurately.

There are two types of the dirt adheres to the protection window 22. Onetype of the dirt is “entire dirt (full cover dirt)” which adheres to anentire surface of the protection window 22 (to fully cover the entiresurface), such as snow, water droplets, snow melting agents. The othertype of the dirt is “partial dirt” adheres to a part of the surface ofthe protection window 22, such as mud. This monitor device performs anentire dirt determination process for determining whether or not a stateof the protection window 22 is an entire dirt state (which is a state inwhich the entire dirt has adhered to the protection window 22). Thismonitor device also performs a partial dirt determination process fordetermining whether or not the state of the protection window 22 is apartial dirt state (which is a state in which the partial dirt hasadhered to the protection window 22).

When it is determined that the state of the protection window 22 is atleast one of the entire dirt state and the partial dirt state, thismonitor device prohibits itself from performing controls based on thecamera image, in other words, the above described alert control and theabove described brake control. Further, this monitor device performs adirt removed determination process, including a “first process fordetermining whether or not the entire dirt is removed so that the entiredirt state ends, the first process being performed when it is (or hasbeen) determined that the state of the protection window 22 is theentire dirt state” and a “second process for determining whether or notthe partial dirt is removed so that the partial dirt state ends, thesecond process being performed when it is (or has been) determined thatthe state of the protection window 22 is the partial dirt state”. Whenit is determined that the state of the protection window 22 is ano-dirt-state through the dirt removed determination process, in otherwords, when all of the dirt states end, the controls based on the cameraimage is allowed to be performed by this monitor device.

Firstly, the entire dirt determination process is described. As shown inFIG. 3, This monitor device divides the camera image CI which has anellipse shape into “a determination area CIc which encompasses/includesa center point Pc of the camera image CI” and an area other than(outside/except) the determination area CIc. This monitor device furtherdivides the determination area CIc into a “center area CA which has arectangle shape encompassing/including the center point Pc” and an“outer area OA which is outside of the center area CA”. The center areaCA may be referred to as a “first area”, and the outer area OA may bereferred to as a “second area”. The determination area CIc is dividedinto a plurality of individual areas AR. Each of the individual areas ARhas a rectangle shape. In the example shown in FIG. 3, the center areaCA includes 20 (twenty) individual areas AR, and the outer area OAincludes 21 (twenty one) individual areas AR.

In general, a part at (in the vicinity of) the center point of thecamera image is the clearest in the camera image. As a distance betweena part of the camera image and the center point of the camera image islonger, the part of the camera image becomes more unclear. In otherwords, a blur degree in the camera image is the smallest at the centerpoint of the camera image, and a blur degree in the camera image isgreater at a point farther away from the center point of the cameraimage. Therefore, in general, edge strength of an area in the vicinityof the center of the camera image is greater than edge strength of anarea in the vicinity of a periphery of the camera image. Thus, the edgestrength of the outer area OA is relatively small, and the edge strengthof the center area CA is relatively great, because the outer area OA isfarther from the center point of the camera image CI than the centerarea CA.

This monitor device executes the entire dirt determination process usingthe above described property of the camera image. Firstly, this monitordevice calculates horizontal direction edge strength ESx of each ofpixels included in (belonging to) each of the individual areas AR inaccordance with an expression 1, and vertical direction edge strengthESy of each of the pixels included in (belonging to) each of theindividual areas AR in accordance with an expression 2. Next, thismonitor device calculates edge strength ES of each of the pixelsincluded in (belonging to) each of the individual areas AR in accordancewith an expression 3.ESx=I(x,y)−I(x−1,y)  (1)ESy=(x,y)−I(x,y−1)  (2)ES=√{square root over (ESx ² +ESy ²)}  (3)

A pixel which is located at the bottom left corner of each of theindividual areas AR is defined as an origin O of x-y coordinates (foreach of the individual areas AR). A horizontal direction of each of theindividual areas AR is defined as an x axis. A vertical direction ofeach of the individual areas AR is defined as a y axis. “I(x, y)” in theabove expressions 1 and 2 represents a pixel value (R, G, B) of thepixel at the coordinates (x, y). The horizontal direction edge strengthESx is a vector which has component values (XR, XG, XB) corresponding tored, green, and blue, respectively. The magnitude of the horizontaldirection edge strength ESx is represented by the following expression.|ESx|=(XR ² +XG ² +XB ²)^(1/2).Similarly, the vertical direction edge strength ESy is a vector whichhas component values (YR, YG, YB) corresponding to red, green, and blue,respectively. The magnitude of the vertical direction edge strength ESyis represented by the following expression.|ESy|=(YR ² +YG ² +YB ²)^(1/2)Subsequently, this monitor device counts the number of pixels(hereinafter referred to as an “edge pixel number”) EN for each of theindividual areas AR, each of the pixels having the edge strength ESwhich is equal to or greater than a threshold strength (a firstthreshold strength) ES1th. This monitor device specifies the individualarea(s) AR, each of which edge pixel number EN is equal to or greaterthan a threshold pixel number (a first threshold pixel number) EN1th.The individual area AR whose edge pixel number EN is equal to or greaterthan the threshold pixel number EN1th is referred to as a “strong edgearea”. This strong edge area is an area where an edge can be detectedclearly.

Subsequently, this monitor device counts the number of the strong edgeareas CEN among the individual areas AR included in (belonging to) thecenter area CA. Hereinafter, the number of the strong edge areas CEN maybe referred to as a “center edge area number”, a “first edge areanumber” or a “first area index value”.

This monitor device determines whether or not the center edge areanumber CEN is smaller than a threshold area number CEN1th. For example,the threshold area number CEN1th is set to “1”.

When the center edge area number CEN is equal to or greater than thethreshold area number CEN1th, this monitor device determines that thestate of the protection window 22 is not the entire dirt state. In theexample shown in FIG. 3, if the dirt has not adhered to the protectionwindow 22, edges of white lines on a road are detected in the individualareas AR1 through AR7 included in the center area CA. Therefore, thismonitor device specifies each of the individual areas AR1 through AR7 asthe strong edge area. In this case, the center edge area number CEN isequal to or greater than the threshold area number CEN1th, because thecenter edge area number CEN is “7 (seven)”. Therefore, this monitordevice determines that the state of the protection window 22 is not theentire dirt state.

On the other hand, when the center edge area number CEN is smaller thanthe threshold area number CEN1th, this monitor device determines thatthe state of the protection window 22 is the entire dirt state, in otherwords, that the state of the protection window 22 is the “state wherethe dirt has adhered to the entire surface of the protection window 22”.No edges is likely to be detected in the center area CA and no edges islikely to be detected in the outer area OA, when the dirt has adhered tothe entire surface of the protection window 22. FIG. 4 shows an exampleof the camera image photographed by the camera 22 when snow has adheredto the entire surface of the protection window 22. In the camera imageshown in FIG. 4, the center edge area number CEN is smaller than thethreshold area number CEN1th, because the center edge area CEN is “0”.Therefore, in this case, this monitor device determines that the stateof the protection window 22 is the entire dirt state.

As described above, the edge strength ES in the outer area OA tends tobe smaller than the edge strength ES in the center area CA. Even if thestate of the protection window 22 is not the entire dirt state, a“possibility that the individual area AR included in the outer area OAis specified as the strong edge area” is low. Therefore, this monitordevice determines whether or not the state of the protection window 22is the entire dirt state, without using the number of the strong edgeareas included in (belonging to) the outer area OA, but through usingthe number of the strong edge areas included in (belonging to) thecenter area CA. Accordingly, a possibility that this monitor device candetermine whether or not the state of the protection window 22 is theentire dirt state accurately can be improved/increased.

In a case where this monitor device has determined that the state of theprotection window 22 is the entire dirt state, this monitor devicedetermines that entire dirt is removed, in other words, that the entiredirt state ends, when at least one of the following conditions (1) or(2) is established. The condition (1) may be referred to as a “firstcondition”, and the condition (2) may be referred to as a “secondcondition”.

Condition (1): the center edge area number CEN is equal to or greaterthan the threshold area number CEN1th.

Condition (2): the number of the strong edge areas included in(belonging to) the outer area OA (hereinafter referred to as an “outeredge area number”, a “second edge area number”, or a “second area indexvalue”) OEN is equal to or greater than a threshold area number OEN1thwhich is a second threshold end value.

The strong edge area in the outer area OA is the individual area AR withthe edge pixel number equal to or greater than a second threshold pixelnumber, the edge pixel number being the number of the edge pixels withthe edge strength ES equal to or greater than a second thresholdstrength. Although in this embodiment, the second threshold strength isset to the same value as the first threshold strength ES1th, in someembodiments, the second threshold strength is set to a value differentfrom the first threshold strength ES1th. In addition, in thisembodiment, the second threshold pixel number is set to the same valueas the first threshold pixel number EN1th, in some embodiments, thesecond threshold pixel number is set to a value different from the firstthreshold pixel number EN1th.

Furthermore, in this embodiment, the threshold area number OEN1th is setto “1”, as with the threshold area number CEN1th, in some embodiments,these threshold area numbers OEN1th and CEN1th are different from eachother.

As described above, the edge strength ES in the outer area OA tends tobe smaller than the edge strength ES in the center area CA. Accordingly,when the outer edge area number OEN in the outer area OA is equal to orgreater than the threshold area number OEN1th, in other words, when thecondition (2) is established, it is considered that the dirt which hadadhered to the entire surface of the protection window 22 has beenremoved. When the outer edge area number OEN is equal to or greater thanthe threshold area number OEN1th, it is considered in general that thecenter edge area number CEN is equal to or greater than the thresholdarea number CEN1th, in other words, that the condition (1) is alsoestablished.

Meanwhile, a case may arise where no edge is detected in the center areaCA and some edges are detected in the center area CA, depending on thescene photographed by the camera 21, as shown in FIG. 5. In the cameraimage shown in FIG. 5, the center edge area number CEN is smaller thanthe threshold area number CEN1th so that the condition (1) is notestablished, because no edge is detected in the center area CA. Incontrast, the outer edge area number OEN is equal to or greater than thethreshold area number OEN1th, because edges of 7 (seven) white lines aredetected in the areas which are located in the outer area OA and abovethe center area CA. In the camera image shown in FIG. 5, the condition(1) is not established, however the condition (2) is established.Therefore, this monitor device can determine that the state of theprotection window 22 is the no-dirt-state (even if the scenephotographed by the camera 21 is such a scene shown in FIG. 5). That is,when only the condition (2) is established, this monitor devicedetermines that the entire dirt state ends.

Next, the partial dirt determination process is described. This monitordevice performs a process for specifying an area where pixel values donot change (remains unchanged) substantially, every time a predeterminedtime period elapses. Further, this monitor device specifies theindividual area AR whose pixel values do not change (remains unchanged)substantially for predetermined processing times as an unchanged areaUCA. When the unchanged area UCA specified by this monitor device ispresent, this monitor device specifies the unchanged area UCA as apartial dirt area to determine that the state of the protection window22 is the partial dirt state.

When this monitor device has determined that the state of the protectionwindow 22 is the partial dirt state, this monitor device determineswhether or not each of the partial dirt areas (the unchanged areas UCA)turns into a dirt removed strong edge area. The dirt removed strong edgearea is the individual area AR with the number of “pixels which isincluded in the partial dirt area and whose edge strength is equal to orgreater than a threshold strength (a third threshold strength)” beingequal to or greater than a threshold pixel number. When all of theindividual areas AR which has been determined as the partial dirt areashave turned into the dirt removed strong edge areas, this monitor devicedetermines that the state of the protection window 22 becomes a statewhere there is no partial dirt (part), in other words, that the partialdirt state ends. Although, in this embodiment, the third thresholdstrength is set to the same value as the first threshold strength ES1th,in some embodiments, the third threshold strength is set to a valuedifferent from the first threshold strength ES1th. In addition, in thisembodiment, the third threshold pixel number is set to the same value asthe first pixel number EN1th, in some embodiments, the third thresholdpixel number is set to a value different from the first threshold pixelnumber EN1th.

As understood from the above example, in the entire dirt determinationprocess, the number of the strong edge areas in the outer area OA wherethe strong edge area is not easy to be detected is not used, but thenumber (the center edge area number CEN) of the strong areas in thecenter area CA where the strong area is easy to be detected is used.That is, when the center edge area number CEN is smaller than thethreshold area number CEN1th, this monitor device determines that thestate of the protection window 22 is the entire dirt state. In thismanner, this monitor device determines whether or not the state of theprotection window 22 is the entire dirt state based on the number (CEN)of the strong edge areas in the center area CA. Therefore, this monitordevice can determine whether or the state of the protection window 22 isthe entire dirt state accurately.

In the case where the state of the protection window 22 is the entiredirty state, this monitor device determines that the entire dirt is (hasbeen) removed when at least one of the conditions (1) and (2) (i.e.condition (1) and/or condition (2)) is established. Thus, when the outeredge area number OEN obtained for the “outer area OA where the edgestrength ES tends to be smaller than the edge strength ES in the centerarea CA” is equal to or greater than the threshold area number OEN1th,this monitor device determines that the entire dirt is (has been)removed, even if the center edge area number CEN is smaller than thethreshold area number CEN1th. Therefore, this monitor device canaccurately determine whether or not the entire dirt is removed. Inaddition, as described above, even if no edge is detected in the centerarea CA of the camera image, this monitor device can accuratelydetermine that the entire dirt is (has been) removed, when the condition(2) is established.

<Specific Operation>

The CPU 11 of the monitor ECU 10 executes a routine represented by aflowchart shown in FIG. 6, every time a predetermined time periodelapses. The routine shown in FIG. 6 is a routine for performing apre-collision control (a pre-collision control when the own vehicle SVruns backward) which is one of the support controls against the obstaclewhich is present in the backward area of the own vehicle SV when the ownvehicle SV runs backward.

When a predetermined timing has come, the CPU 11 starts the process fromStep 600 shown in FIG. 6, and proceeds to Step 605 to acquire the shiftposition from the shift position sensor 25. Subsequently, the CPU 11proceeds to Step 610 to determine whether or not the shift positionacquired at Step 605 is the reverse range (“R”). When the shift positionis not “R”, in other words, when the shift position is the drive range(“D”), the neutral range (“N”), or the like, the CPU 11 makes a “No”determination at Step 610. Thereafter, the CPU 11 proceeds to Step 695so as to tentatively terminate the present routine. As a result, thepre-collision control is not performed.

On the other hand, when the shift position is “R”, the CPU 11 makes a“Yes” determination at Step 610, and proceeds to Step 615. At Step 615,the CPU 11 acquires the vehicle state information from the vehicle statesensor 26, and proceeds to Step 620.

At Step 620, the CPU 11 predicts a predicted travel path RCR (refer toFIG. 2) along which the own vehicle SV will travel, based on the vehiclestate information acquired at Step 615, and proceeds to Step 625.

The process at Step 620 is described more specifically with reference toFIG. 2.

The CPU 11 calculates a turning radius of the own vehicle SV based on“the vehicle velocity Vs of the own vehicle SV and the yaw rate Yr”included in the vehicle state information acquired at Step 615.Thereafter, the CPU 11 predicts, as the predicted travel path RCR, atravel path along which “the center point PO (refer to FIG. 2) of awheel axis connecting a rear left wheel and a rear right wheel” willmove, based on the calculated turning radius. When the yaw rate Yr isgenerated (nonzero), the CPU 11 predicts an arc travel path as thepredicted travel path RCR. When the yaw rate is “0”, the CPU 11 predictsa straight travel path along a direction of the acceleration of the ownvehicle SV as the predicted travel path RCR.

Referring back to FIG. 6, at Step 625, the CPU 11 acquires the objectinformation from the clearance sonars 24, and proceeds to Step 630. AtStep 630, the CPU 11 determines whether or not both a value of an entiredirt flag Xz and a value of a partial dirt flag Xb are “0”. The value ofthe entire dirt flag Xz is set to “1” when it is determined that thestate of the protection window 22 is the entire dirt state through theprocess described later. In a case where the value of the entire dirtflag Xz is “1”, the value of the entire dirt flag Xz is set to “0” whenit is determined that the entire dirt state ends through the processdescribed later. The value of the partial dirt flag Xb is set to “1”when it is determined that the state of the protection window 22 is thepartial dirt state through the process described later. In a case wherethe value of the partial dirt flag Xb is “1”, the value of the partialdirt flag Xb is set to “0” when it is determined that the partial dirtstate ends through the process described later. Therefore, at Step 630,the CPU 11 substantially determines whether or not “no-dirt-informationrepresenting that the state of the protection window 22 is theno-dirt-state” has been stored in the RAM 13.

When at least one of the value of the entire dirt flag Xz and (or) thevalue of the partial dirt flag Xb is “1”, in other words, when at leastone of “entire dirt information representing that the state of theprotection window 22 is the entire dirt state” and “partial dirtinformation representing that the state of the protection window 22 isthe partial dirt state” has been stored in the RAM 13, the CPU 11 makesa “No” determination at Step 630, and proceeds to Step 695 totentatively terminate the present routine. When the state of theprotection window 22 is at least one of the entire dirt state and (or)the partial dirt state, the object is unlikely to be detectedaccurately, because the dirt is displayed on the camera imagephotographed through the protection window 22. In this case, the CPU 11may perform the pre-collision control incorrectly. In view of the above,in this case, the CPU 11 prohibits itself from performing (i.e., doesnot perform) the pre-collision control.

On the other hand, when both the value of the entire dirt flag Xz andthe value of the partial dirt flag Xb are “0”, that is, when theno-dirt-information has been stored in the RAM 13, the CPU 11 makes a“Yes” determination at Step 630, and proceeds to Step 635 to acquire thecamera image information from the camera system 20. Subsequently, theCPU 11 proceeds to Step 640. At Step 640, the CPU 11 fuses the objectinformation acquired at Step 625 with the camera image informationacquired at Step 635, to specify the location(s) of the object(s) inrelation to the own vehicle SV.

Thereafter, the CPU 11 proceeds to Step 645 to select, as the obstacle,the object which has the high probability of colliding with the ownvehicle SV or which is predicted to excessively/extremely approaches theown vehicle SV, among the objects whose locations are specified at Step640, based on the predicted travel path RCR predicted at Step 620, thelocation of the object specified at Step 640, and the relative velocityof the object.

The process at Step 645 is described more specifically with reference toFIG. 2.

The CPU 11 predicts, based on the “predicted travel path RCR” with afinite length, a predicted left travel path LEC along which a point PLwill move and a predicted right travel path REC along which a point PRwill move. The point PL is a point positioned leftward by apredetermined distance αL from a left end of a body of the own vehicleSV. The point PR is a point positioned rightward by a predetermineddistance αR from a right end of the body of the own vehicle SV. That is,the predicted left travel path LEC is a path obtained by parallellyshifting the predicted traveling path RCR to the left direction of theown vehicle SV by a “distance obtained by adding a half of avehicle-body width to the predetermined distance αL”. The predictedright travel path REC is a path obtained by parallelly shifting thepredicted travel path RCR to the right direction of the own vehicle SVby a “distance obtained by adding a half of the vehicle-body width tothe predetermined distance αR”. Each of the distance αL and the distanceαR is a distance which is equal to or longer than “0”. The distance αLand the distance αR may be the same as each other, or may be differentfrom each other. The CPU 11 specifies/designates, as a predicted travelpath area ECA, an area between the predicted left travel path LEC andthe predicted right travel path REC.

Thereafter, the CPU 11 calculates/predicts a moving trajectory of theobject based on the past locations/positions of the object. The CPU 11calculates/predicts a moving direction of the object in relation to theown vehicle SV, based on the calculated moving trajectory of the object.Subsequently, the CPU 11 selects/extracts, as the obstacle(s) which hasa probability (high probability) of colliding with the own vehicle SV,

one or more of the objects which has been in the predicted travel patharea ECA and which will intersect with a rear end area TA of the ownvehicle SV, and

one or more of the objects which will be in the predicted traveling patharea ECA and which will intersect with the rear end area TA of the ownvehicle SV,

based on the predicted traveling path area ECA, the relativerelationships (the relative locations and the relative velocities) ofthe objects in relation to the own vehicle SV, and the moving directionsof the objects in relation to the own vehicle SV. The rear end area TAis an area represented by a line segment between the point PL and thepoint PR.

The CPU 11 predicts the “trajectory/path along which the point PL willmove” as the predicted left travel path LEC, and predicts the“trajectory/path along which the point PR will move” as the predictedright travel path REC. If both of the values αL and αR are positivevalues, the CPU 11 determines the “object which has been in thepredicted travel path area ECA and will intersect with the rear end areaTA” or the “object which will be in the predicted travel path area ECAand will intersect with the rear end area TA”, as the object withprobability of passing near the left side or the right side of the ownvehicle SV.” Accordingly, the CPU 11 can select/extract, as theobstacle, the object with the probability of passing near the left sideor the right side of the own vehicle SV.

Referring back to FIG. 6, after the CPU 11 executes the process at Step645, the CPU 11 proceeds to Step 650 to determine whether or not the CPU11 has selected any one of the objects as the obstacle at Step 645. Whenthe CPU 11 has selected none of the objects as the obstacle at Step 645,the CPU 11 makes a “No” determination at Step 650, and proceeds to Step695 to tentatively terminate the present routine. As a result, thepre-collision control is not performed. On the other hand, when the CPU11 has selected any one of the objects as the obstacle at Step 645, theCPU 11 makes a “Yes” determination at Step 650, and proceeds to Step655.

At Step 655, the CPU 11 calculates, for each of the obstacle(s), thetime to collision TTC which it takes for each of the obstacle(s) tointersect with the rear end area (refer to FIG. 2) of the own vehicleSV, and proceeds to Step 660.

The process at Step 655 is described more specifically.

The CPU 11 calculates the time to collision TTC of the obstacle throughdividing the distance between the own vehicle SV and the obstacle by therelative velocity of the obstacle in relation to the own vehicle SV.

The time to collision TTC is either a time period T1 or a time periodT2, described below.

The time period T1 is a time period which it takes for the obstacle tocollide with the own vehicle SV (a time period from the present timepoint to a predicted collision time point).

The time period T2 is a time period which it takes for the obstaclewhich has probability of passing near either side of the own vehicle SVto reach the closest point to the own vehicle SV (a time period from thepresent time point to the time point when the obstacle most closelyapproaches the own vehicle SV).

The time to collision TTC is a time period which it takes for theobstacle to reach the “rear end area TA of the own vehicle SV” under anassumption that the obstacle and the own vehicle SV move with keepingthe relative velocity and the relative moving direction at the presenttime.

Further, the time to collision TTC represents a time period which ittakes for this monitor device to be allowed/able to perform thepre-collision control or a time period which it takes for the driver tobe allowed/able to perform a collision preventing operation forpreventing the collision between the obstacle and the own vehicle SV.The time to collision TTC is an index value indicative of a collisionprobability of the collision. As the time to collision TTC is shorter,the collision probability is greater/higher. As the time to collisionTTC is longer, the collision probability is smaller/lower.

At Step 660, the CPU 11 determines whether or not each of the times tocollision TTC calculated at Step 655 is equal to or shorter than atleast one of time thresholds T(n)th. When all of the times to collisionTTC is longer than any of the time thresholds T(n)th, the CPU 11 makes a“No” determination at Step 660, and proceeds to Step 695 to tentativelyterminate the present routine. In contrast, when at least one of thetimes to collision TTC is equal to or shorter than at least one of thetime thresholds T(n)th, the CPU 11 makes a “Yes” determination at Step660, and proceeds to Step 665 to perform the pre-collision controlcorresponding to the time threshold T(n)th which the time to collisionTTC is equal to or shorter than. Thereafter, the CPU 11 proceeds to Step695 to tentatively terminate the present routine.

In the present example, the time thresholds T(n)th includes the timethreshold T1th for the alert control and the time threshold T2th for thebrake control. The time threshold T1th is longer than the time thresholdT2th. When any one of the times to collision TTC is equal to or shorterthan the time threshold T1th, the CPU 11 transmits the displayinstruction signal to the display unit 40 so as to display the alertscreen on the display unit 30, and transmits the output instructionsignal to the speaker 31 so as to output the alert sound from thespeaker 31. When any one of the times to collision TTC is equal to orshorter than the time threshold T2th, the CPU 11 selects the obstaclewith the minimum time to collision TTC among the obstacles with the timeto collision TTC which is equal to or shorter than the time thresholdT2th. The CPU 11 calculates a deceleration (the target deceleration TG)which is required to stop the own vehicle SV before the own vehicle SVcollides with the selected obstacle, based on the relative velocity ofthe selected obstacle and the location of the selected obstacle.Thereafter, the CPU 11 transmits the brake instruction signal includingthe target deceleration TG to the brake ECU 32, and transmits the torquedecreasing instruction signal to the engine ECU 35. As a result, the ownvehicle SV decelerates at a deceleration which is approximately the sameas the target deceleration TG.

The CPU 11 executes a routine represented by a flowchart shown in FIG.7, every time a predetermined time period elapses. The routine shown inFIG. 7 is a routine for determining whether or not the state of theprotection window 22 is the entire dirt state.

When a predetermined timing has come, the CPU 11 starts the process fromStep 700 shown in FIG. 7, and proceeds to Step 705 to determine whetheror not the value of the entire dirt flag Xz is “0”, in other words,whether or not the entire dirt information has been stored in the RAM13. When the value of the entire dirt flag Xz is “1”, that is, when ithas been determined that the state of the protection window 22 is theentire dirt state, the CPU 11 makes a “No” determination at Step 705,and proceeds to Step 795 so as to tentatively terminate the presentroutine.

On the other hand, when the value of the entire dirt flag Xz is “0”,that is, when the entire dirt information has not been stored in the RAM13, the CPU 11 makes a “Yes” determination at Step 705, and executesprocesses from Step 710 to Step 735 in order so as to proceed to Step740.

Step 710: The CPU 11 acquires the camera image information from thecamera system 20.

Step 715: The CPU 11 divides the camera image included in (representedby) the camera image information into the individual areas AR (refer toFIG. 3).

Step 720: The CPU 11 separates the individual areas AR into theindividual areas AR included/encompassed in the center area CA (refer toFIG. 3) and the individual areas AR included/encompassed in the outerarea OA (refer to FIG. 3).

Step 725: The CPU 11 calculates the edge strength of each of the pixelsincluded in the individual areas AR encompassed in the center area CA,in accordance with the above expressions 1 through 3.

Step 730: The CPU 11 counts the edge pixel number EN representing thenumber of pixels, each of the pixels having the edge strength equal toor greater than the threshold strength ES1th, in/for each of theindividual areas AR encompassed in the center area CA.

Step 735: The CPU 11 counts the number of the individual areas AR (thecenter edge area number CEN) with/having the edge pixel number EN equalto or greater than the threshold pixel number EN1th.

At step 740, the CPU 11 determines whether or not the center edge areanumber CEN counted at Step 735 is smaller than the threshold area number(the first threshold area number) CEN1th. When the center edge areanumber CEN is equal to or greater than the threshold area number CEN1th,it cannot be determined that the state of the protection window 22 isthe entire dirt state. Therefore, when the center edge area number CENis equal to or greater than the threshold area number CEN1th, the CPU 11makes a “No” determination at Step 740, and directly proceeds to Step795 so as to tentatively terminate the routine. As a result, the entiredirt information is not stored in the RAM 13.

On the other hand, when the center edge area number CEN is smaller thanthe threshold area number CEN1th, it can be determined that the state ofthe protection window 22 is the entire dirt state. Therefore, when thecenter edge area number CEN is smaller than the threshold area numberCEN1th, the CPU 11 makes a “Yes” determination at Step 740, and proceedsto Step 745 so as to set the value of the entire dirt flag Xz to “1”.That is, the CPU 11 stores the entire dirt information into the RAM 13.Thereafter, the CPU 11 proceeds to Step 795 so as to tentativelyterminate the present routine.

As understood from the above example, when the center edge area numberCEN is smaller than the threshold area number CEN1th, the CPU 11determines that the dirt has adhered to (or the dirt is present on) theentire surface of the protection window 22, so as to set the value ofthe entire dirt flag Xz to “1”. The CPU 11 can determine whether or notthe dirt has adhered to the entire surface of the protection window 22based on the edge area number in the center area CA where the edgestrength ES tends to be accurately calculated to be a valuesubstantially equal to the true/inherent value. Therefore, the CPU 11can determine whether or not the dirt has adhered to the entire surfaceof the protection window 22 accurately.

The CPU 11 executes a routine represented by a flowchart shown in FIG.8, every time a predetermined time period elapses. The routine shown inFIG. 8 is a routine for determining whether or not the state of theprotection window 22 is the partial dirt state.

When a predetermined timing has come, the CPU 11 starts the process fromStep 800 shown in FIG. 8, and proceeds to Step 805 so as to acquire thevehicle state information from the vehicle state sensor 26.

Subsequently, the CPU 11 proceeds to Step 810 to determine whether ornot the magnitude of the vehicle velocity Vs included in the vehiclestate information acquired at Step 805 is greater than “0 m/s”, in otherwords, whether or not the own vehicle SV is traveling.

When the magnitude of the vehicle velocity Vs is “0 m/s”, the CPU 11makes a “No” determination at Step 810, and proceeds to Step 895 so asto tentatively terminate the present routine. When the magnitude of thevehicle velocity Vs is “0 m/s”, in other words, when the own vehicle SVstops, a probability that a present camera image Fn acquired at thepresent time point remains unchanged from a previous camera image Fn−1acquired at the previous time point (i.e., a time point thepredetermined time period before the present time point) is high,regardless of whether or not the partial dirt has adhered to theprotection window 22. Therefore, when the magnitude of the vehiclevelocity Vs is “0 m/s”, the CPU 11 tentatively terminates the presentroutine so as not to execute processes of Steps at and after Step 815.

On the other hand, the magnitude of the vehicle velocity Vs is greaterthan “0 m/s”, the CPU 11 makes a “Yes” determination at Step 810, andexecutes the processes from Step 815 to Step 830 in order so as toproceed to Step 835.

Step 815: The CPU 11 acquires the camera image information from thecamera system 20 as the present camera image Fn. The “camera imageacquired at Step 815 which was executed at the time point before thepresent time point by the predetermined time period” is referred to as aprevious camera image Fn−1.

Step 820: The CPU 11 generates/produces a difference image Sn(Sn=Fn−Fn−1) between the present camera image Fn and the previous cameraimage Fn−1.

More specifically, the CPU 11 calculates a subtraction value bysubtracting a pixel value of each of the pixels included in the previouscamera image Fn−1 from a pixel value of the corresponding one of thepixels included in the present camera image Fn. Thereafter, the CPU 11acquires the magnitude of the subtraction value of each of the pixels,as a pixel value of the corresponding one of the pixels included in thedifference image Sn.

Step 825: The CPU 11 adds the difference image Sn to an integrateddifference image stored in a specific part (hereinafter, referred to asan “integration memory”) in the RAM 13 to obtain an integration result,and stores the integration result into the integration memory as a newintegrated difference image. As a result, a total value (hereinafter,referred to as an “integration value”) VI is calculated. The integrationvalue VI represents a total of a magnitude of change of/in the pixelvalue of each of the pixels included in the camera image in a periodfrom a time point when the integration memory was initialized to thepresent time point

Step 830: The CPU 11 adds “1” to an integration times counter AC” toupdate the integration times counter AC. The integration times counterAC represents the number of times of integrating the difference imageSn.

Subsequently, the CPU proceeds to Step 835 to determine whether or notthe value of the integration times counter AC is equal to or greaterthan a threshold counter value AC1th. Although, in the present example,the threshold counter value AC1th is set to “1”, in some embodiments,the threshold counter value AC1th is set to a value other than “1”. Whenthe value of the integration times counter AC is smaller than thethreshold counter value AC1th, the CPU 11 makes a “No” determination atStep 835, and directly proceeds to Step 895 so as to tentativelyterminate the present routine. On the other hand, when the value of theintegration value AC is equal to or greater than the threshold countervalue AC1th, the CPU 11 makes a “Yes” determination at Step 835, andproceeds to Step 840.

At Step 840, the CPU 11 divides the difference image Sn into theindividual areas AR so as to calculate the number of the pixels(hereinafter, referred to as an “unchanged pixel number”) UCPN, each ofthe pixels having the integration value IV equal to or smaller than athreshold integration value (a changed amount determination value)IV1th, in/for each of the individual areas AR.

Subsequently, the CPU 11 proceeds to Step 845 to select the individualarea AR (hereinafter, referred to as an “unchanged area UCA”)with/having unchanged pixel number UCPN equal to or greater than athreshold pixel number UCPN1th. This unchanged area UCA can be expressedas the individual area AR where “density of the pixels (hereinafter,referred to as “unchanged pixels”) with/having the integration value IVequal to or smaller than the threshold integration value IV1th” is equalto or higher than a threshold density.

Subsequently, the CPU 11 proceeds to Step 850 so as to determine whetheror not the unchanged area UCA has been selected at Step 845. When theunchanged area UCA has been selected at Step 845, it can be determinedthat the partial dirt has adhered to the protection window 22.

In view of the above, when the unchanged area UCA has been selected, theCPU 11 makes a “Yes” determination at Step 850, executes the processesfrom Step 855 to Step 865 in order, and proceeds to Step 895 so as totentatively terminate the present routine.

Step 855: The CPU 11 sets the value of the partial dirt flag Xb to “1”.That is, the CPU 11 stores the partial dirt information into the RAM 13.At this time point, the CPU 11 stores “information to identify whichindividual area AR is the unchanged area UCA selected at Step 845” intothe RAM 13.

When the value of the partial dirt flag Xb has already been set to “1”at the time point at which the CPU 11 executes the process at Step 855,the CPU 11 sets the value of the partial dirt flag Xb to “1” again, andstores “identification information of the unchanged area UCA which hasbeen selected at Step 845” in the RAM 13.

Step 860: The CPU 11 deletes the difference image Sn stored in theintegration memory so as to initialize the integration memory.

Step 865: The CPU 11 sets the value of the integration times counter ACto “0” so as to initialize the integration times counter AC.

In contrast, when the unchanged area UCA has not been selected at thetime point at which the CPU 11 executes the process at Step 850, the CPU11 makes a “No” determination at Step 850, and proceeds to the processesof Step 860 and thereafter.

As understood from the above example, when the CPU 11 has detected atleast one of the individual areas AR where the “number of the pixelseach of which has the pixel value that remains unchanged substantiallyfor/over the predetermined time period” is equal to or greater than athreshold number, the CPU 11 determines that the state of the protectionwindow 22 is the partial dirt state. This individual area AR is theunchanged area UCA having the unchanged pixel number UCPN which is equalto or greater than the threshold pixel number UCPN1th. Therefore, theCPU 11 can detect the dirt (for example, mud) which has adhered to apart of the surface of the protection window 22 as the partial dirt.

The CPU 11 executes a routine represented by a flowchart shown in FIG.9, every time a predetermined time period elapses. The routine shown inFIG. 9 is a routine for determining whether or not the entire dirt whichhas adhered to the protection window 22 has been removed and the partialdirt which has adhered to the protection window 22 has been removed.

When a predetermined timing has come, the CPU 11 starts the process fromStep 900 shown in FIG. 9, executes the processes from Step 905 to Step925 in order, and proceeds to Step 930.

Step 905: The CPU 11 acquires the camera image information from thecamera system 20.

Step 910: The CPU 11 divides the camera image into the individual areasAR, as with Step 715 shown in FIG. 7.

Step 915: The CPU 11 separates the individual areas AR into theindividual areas AR included/encompassed in the center area CA and theindividual areas AR included/encompassed in the outer area OA, as withStep 720 shown in FIG. 7.

Step 920: The CPU 11 calculates the edge strength of each of the pixelsincluded in the individual areas AR, in accordance with the aboveexpressions 1 through 3.

Step 925: The CPU 11 counts the edge pixel number EN representing thenumber of pixels, each of the pixels having the edge strength ES equalto or greater than the threshold strength ES1th, in/for each of theindividual areas AR.

Subsequently, the CPU 11 proceeds to Step 930 to determine whether ornot the value of the entire dirt flag Xz is “1”. That is, the CPU 11determines whether or not the entire dirt information has been stored inthe RAM 13. When the value of the entire dirt flag Xz is “1”, the CPU 11makes a “Yes” determination at Step 930, and proceeds to Step 935.

At Step 935, the CPU 11 counts the number of the individual areas AR(the center edge area number CEN) which is encompassed/included in thecenter area CA and which has the edge pixel number EN equal to orgreater than the threshold pixel number EN1th, as with Step 735 shown inFIG. 7. Subsequently, the CPU 11 proceeds to Step 940 to determinewhether or not the center edge area number CEN is smaller than thethreshold area number CEN1th.

When the center area edge area number CEN is equal to or greater thanthe threshold area number CEN1th, the CPU 11 determines that the entiredirt which has adhered to the protection window 22 has been removed. Inthis case, the CPU 11 makes a “No” determination at Step 940, andproceeds to Step 945 to set the value of the entire dirt flag Xz to “0”.That is, the CPU 11 deletes the entire dirt information stored in theRAM 13. Thereafter, the CPU 11 proceeds to Step 950.

At Step 950, the CPU 11 determines whether or not the value the partialdirt flag Xb is “1”. That is, the CPU 11 determines whether or not thepartial dirt information has been stored in the RAM 13. When the valueof the partial dirt flag Xb is “1”, the CPU 11 makes a “Yes”determination at Step 950, and proceeds to Step 960.

At Step 960, the CPU 11 determines whether or not the “edge pixel numberEN in each of the individual areas AR which has been selected as theunchanged area UCN at Step 845” is equal to or greater than thethreshold pixel number EN1th. That is, the CPU 11 determines whether ornot all of the unchanged areas UCA are (or have turned into) the strongedge areas (the individual area where the edges are detected clearly).

When a determination condition of Step 960 is established, the CPU 11determines that all partial dirt which has adhered to the protectionwindow 22 has been removed. In this case, the CPU 11 makes a “Yes”determination at Step 960, and proceeds to Step 965 so as to set thevalue of the partial dirt flag Xb to “0”. That is, the CPU 11 deletesthe partial dirt information stored in the RAM 13. Thereafter, the CPU11 proceeds to Step 995 to tentatively terminate the present routine.

On the other hand, at the time point at which the CPU 11 proceeds toStep 930, the CPU 11 makes a “No” determination at Step 930 if the valueof the entire dirt flag Xz is not “1” (in other words, the entire dirtinformation has not been stored in the RAM 13), to directly proceed toStep 950.

At the time point at which the CPU 11 proceeds to Step 940, the CPU 11makes a “Yes” determination at Step 940 if the center edge area numberCEN is smaller than the threshold area number CEN1th, and proceeds toStep 970. At Step 970, the CPU 11 counts the outer edge area number OEN,and proceeds to Step 975. The outer edge area number OEN represents thenumber of the individual areas AR which are encompassed/included in theouter area OA and each of which has the edge pixel number EN equal to orgreater than the threshold pixel number EN1th.

At Step 975, the CPU 11 determines whether or not the outer edge areanumber OEN is smaller than the threshold area number (the secondthreshold area number) OEN1th. In this example, the threshold areanumber OEN1th is set to “1”. When the outer edge area number OEN isequal to or greater than the threshold area number OEN1th, that is, whenthere is the individual area AR in which the edge is detected clearlyand which is included/encompassed in the outer area where it is not easyfor the edge to be dearly detected, the CPU 11 determines that theentire dirt which has adhered to the protection window 22 has beenremoved. When the outer edge area number OEN is equal to or greater thanthe threshold area number OEN1th, the CPU 11 makes a “No” determinationat Step 975, and proceeds to Step 945 so as to set the value of theentire dirt flag Xz to “0”. In contrast, when the outer edge area numberOEN is smaller than the threshold area number OEN1th, the CPU 11 makes a“Yes” determination at Step 975, and directly proceeds to Step 950.

When the value of the partial dirt flag Xb is “0”, that is, when thepartial dirt value has not been stored in the RAM 13, at the time pointat which the CPU 11 proceeds to Step 950, the CPU 11 makes a “No”determination at Step 950, and directly proceeds to Step 995 so as totentatively terminate the present routine.

At the time point at which the CPU 11 proceeds to Step 960, if there isat least one of the individual area(s) AR which has the edge pixelnumber EN smaller than the threshold pixel number EN1th among theindividual area(s) which has/have been selected as the unchanged area(s)UCA at Step 845, the CPU 11 determines that the partial dirt which hasadhered to the protection window 22 has not been removed yet. In thiscase, the CPU 11 makes a “No” determination at Step 960, and directlyproceeds to Step 995 so as to tentatively terminate the present routine.

As understood from the above example, in the case where the CPU 11 hasdetermined that the entire dirt has adhered to the protection window 22,the CPU 11 determines that the entire dirt has been removed when atleast one of the condition (1) and (or) the condition (2) isestablished, wherein the condition (1) is a condition that the centeredge area number CEN is equal to or greater than the threshold areanumber CEN1th, and the condition (2) is a condition that the outer edgearea number OEN is equal to or greater than the threshold area numberOEN1th. Especially, the condition (2) is established, when the number(the outer edge area number OEN) of the strong edge areas encompassed inthe outer area OA where the edge strength EA tends to be calculated as asmaller value, the strong edge area being an area where an edge isdetected dearly. Therefore, when the entire dirt has been actuallyremoved, the CPU 11 can accurately determine that the entire dirt hasbeen removed based on the camera image even if that camera image is animage where no edge is to be detected in the center area CA.

In the case where the CPU 11 has determined that the partial dirt hasadhered to the protection window 22, the CPU 11 determines that thepartial dirt has been removed when the “condition that the edge pixelnumber EN is equal to or greater than the threshold pixel number EN1th”is established in all of the individual areas AR which have beenselected as the partial dirt areas (the unchanged areas UCA). The CPU 11determines whether or not the partial dirt has been removed withoutusing the areas except the unchanged area UCA, and therefore, the CPU 11can accurately determine whether or not the partial dirt has beenremoved.

As described above, the removal determination condition to be satisfiedwhen determining that the entire dirt is removed has been different fromthe removal determination condition to be satisfied when determiningthat the partial dirt has been removed. In this manner, the CPU 11sets/uses the removal determination condition corresponding to each typeof the dirt which has adhered to the protection window 22 so that theCPU 11 can accurately determine that the dirt has been removed(regardless of the type of the dirt).

The present invention is not limited to the above-described embodiment,and can adopt various modifications of the present invention. In someembodiments, this monitor device sets the value of the entire dirt flagXz to “1” at a timing at which a driver performs an operation to changea position of an ignition key switch of the own vehicle SV from an offposition to an on position. This timing is referred to as a “timing ofan initial state” or an “ignition start timing”. Snow, frost, waterdroplets, or the like, may adhere to the protection window 22 in aperiod from a “time point at which the driver performs an operation tochange the position of the ignition key switch from the on position tothe off position so as to park the own vehicle SV” to a “time point atwhich the state of the own vehicle SV turns into the initial state”, sothat the dirt may adhere to the entire surface of the protection window22. This monitor device can prohibit itself from performing thepre-collision control based on the camera image which may be inaccurateuntil it is determined that the entire dirt ends after the initialstate.

In some embodiments, this monitor device has a temperature sensor formeasuring temperature of the outside of the own vehicle SV (outsidetemperature). In this case, this monitor device acquires the outsidetemperature from the temperature sensor in the initial state. When theoutside temperature is equal to or lower than a threshold temperature,this monitor device sets the value of the entire dirt flag Xz to “1”.When the snow or the frost is likely to adhere to the protection window22 in the initial state, this monitor device can prohibit itself fromperforming the pre-collision control based on the camera image which maybe inaccurate.

In some embodiments, the threshold area number CEN1th used at Step 940is different from the threshold area number OEN1th used at Step 975. Insome embodiments, the threshold area number CEN1th is smaller than thethreshold area number OEN1th, because the edge strength ES in the centerarea CA is calculated more accurately than the edge strength ES in theouter area OA. According to this embodiment, this monitor device canaccurately determine that the entire dirt has been removed. In someembodiments, the threshold area number OEN1th is smaller than thethreshold area number CEN1th, because the edge strength ES in the outerarea OA tends to be calculated to be a relatively small value, ascompared with the edge strength ES in the center area CA. According tothis embodiment, this monitor device can determine that the entire dirthas been removed at an earlier stage.

In some embodiments, the threshold area number CEN1th used for thedetermination as to whether or not the state of the protection window 22is the entire dirt state (the threshold area number CEN1th used at Step740) is different from the threshold area number CEN1th used for thedetermination as to whether or not the entire dirt state ends (thethreshold area number CEN1th used at Step 940). In this case,preferably, the threshold area number CEN1th used for the determinationas to whether or not the entire dirt state ends is greater than thethreshold area number CEN1th used for the determination as to whether ornot the state of the protection window 22 is the entire dirt state.

In some embodiments, this monitor device calculates an average AVE ofthe edge strength ES in/for each of the individual areas AR instead ofthe process of Step 730. Thereafter, this monitor device counts, as thecenter edge area number CEN, the number of the individual areas AR, eachof which is encompassed/included in the center area CA and has theaverage AVE equal to or greater than a threshold average AVE1th insteadof the process of Step 735. Further, this monitor device counts, as thecenter edge area number CEN, the number of the individual areas AR, eachof which is encompassed/included in the center area CA and has theaverage AVE equal to or greater than a threshold average AVE1th insteadof the process of Step 935 shown in FIG. 9. Further, this monitor devicecounts, as a partial dirt edge number DEN, the number of the individualareas AR, each of which is encompassed/included in the unchanged areaUCA and has the average AVE equal to or greater than the thresholdaverage AVE1th instead of the process of Step 955. Thereafter, thismonitor device counts, as the outer edge area number OEN, the number ofthe individual areas AR, each of which is encompassed/included in theouter area OA and has the average AVE equal to or greater than thethreshold average AVE1th instead of the process of Step 970.

In some embodiments, when at least one of the entire dirt or the partialdirt has adhered to an unillustrated protection window of anunillustrated front camera which photographs a scene in front of the ownvehicle SV and which is arranged in the front end of the own vehicle SV,this monitor device prohibits itself from performing a “pre-collisioncontrol when the own vehicle SV travels/runs forward” which is one ofthe support controls. In this embodiment, millimeter-wave radars arearranged at the center in the width direction of the front end of theown vehicle SV. Each of the radar sensors radiates a radio wave in amillimeter waveband (hereinafter referred to as “millimeter wave”).Thereafter, each of the radar sensors receives the reflected wave, anddetects the location of the object in relation to the own vehicle SV andthe relative velocity of the object in relation to the own vehicle SV.If the location of the object can be detected based on the front camera,the monitor device needs not to comprise the millimeter wave radars.This monitor device specifies the location of the object in front of theown vehicle SV based on the detection result of the millimeter waveradar and a camera image (hereinafter referred to as a “front cameraimage”) photographed by the front camera. A processes for performing thepre-collision control when the own vehicle SV travels/runs forwarddiffers from the processes of the routine shown in FIG. 6 only inperforming the pre-collision control to the object in front of the ownvehicle SV (instead of the object in the backward area of the ownvehicle SV).

An entire dirt determination process for the protection window of thefront camera is the same as the routine shown in FIG. 7, a partial dirtdetermination process for this protection window is the same as theroutine shown in FIG. 8, and a process for determining whether or notthe entire dirt and/or the partial dirt has adhered to this protectionwindow is the same as the process of the routine shown in FIG. 9.

When the monitor device is configured to obtain/acquire the location ofthe object in relation to the own vehicle SV based on the camera imagephotographed by the camera 21, the monitor device does not necessarilyhave to comprise the clearance sonars 24.

In some embodiments, the clearance sonars 24 are sensors which radiatewireless media and receive the reflected wireless media so as to detectthe object. Millimeter wave radars, infrared radars, or the like, areused in place of the clearance sonars 24.

The display unit 30 is not limited to the HUD. The display unit 30 maybe a Multi Information Display (MID), a touch panel of the navigationdevice, or the like. The MID is a display panel which is arranged on adash board and which includes a speed meter, a taco meter, a fuel gauge,an water temperature gauge, an od/trip meter, an warning lump, and thelike.

What is claimed is:
 1. A monitor device comprising: an electroniccontrol unit (ECU) configured to: perform a support control to supportdriving of an own vehicle based on a camera image which includes pixelsand which is acquired by a camera for photographing an area around theown vehicle through a protection window which is exposed to outside ofthe own vehicle; determine whether or not a protection window state isan entire dirt state where dirt has adhered to an entire surface of theprotection window, based on edge strengths of the pixels; and determinewhether or not the entire dirt state ends based on the edge strengths,when it has been determined that the protection window state has beenthe entire dirt state, wherein, the ECU is configured not to perform thesupport control, when it has been determined that the protection windowstate has been the entire dirt state, and the ECU is further configuredto: determine that the protection window state is the entire dirt state,when a first area index value calculated based on the edge strengths ofpixels included in a first area is smaller than a threshold dirt value,the first area encompassing a center of the camera image; determine thatthe entire dirt state ends, when a first condition is established, thefirst condition being a condition that the first area index value isequal to or greater than a first threshold end value, and determine thatthe entire dirt state ends, when a second condition is established, thesecond condition being a condition that a second area index valuecalculated based on the edge strengths of pixels included in a secondarea other than the first area in the camera image is equal to orgreater than a second threshold end value, the second area surroundingthe first area on an outer periphery thereof, and the edge strengths ofthe pixels in the first area are greater than the edge strengths of thepixels in the second area.
 2. The monitor device according to claim 1,wherein the ECU is further configured to: calculate, as the first areaindex value, a value correlating with the number of pixels included inthe first area, each of the pixels in the first area having an edgestrength which is equal to or greater than a first threshold strength;and calculate, as the second area index value, a value correlating withthe number of pixels included in the second area, each of the pixels inthe second area having an edge strength which is equal to or greaterthan a second threshold strength.
 3. The monitor device according toclaim 2, wherein the first area is divided into a plurality ofindividual areas; and wherein the ECU is further configured to:calculate, as the first area index value, a first edge area numberindicative of the number of individual areas, each of the individualareas satisfying a condition that the number of pixels included in eachof the individual areas is equal to or greater than a first thresholdpixel number, each of the pixels having the edge strength equal to orgreater than the first threshold strength; and determine that theprotection window state is the entire dirt state, when the first edgearea number is smaller than a first threshold area number serving as thethreshold dirt value.
 4. The monitor device according to claim 2,wherein the second area is divided into a plurality of individual areas;and wherein the ECU is further configured to: calculate, as the secondarea index value, a second edge area number indicative of the number ofindividual areas, each of the individual areas satisfying a conditionthat the number of pixels included in each of the individual areas isequal to or greater than a second threshold pixel number, each of thepixels having the edge strength equal to or greater than the secondthreshold strength; and determine that the second condition isestablished, when the second edge area number is equal to or greaterthan a second threshold area number serving as the second threshold endvalue.
 5. The monitor device according to claim 1, wherein the firstarea and the second area are divided into a plurality of individualareas; and wherein the ECU is further configured to: determine that theprotection window state is a partial dirt state where the dirt hasadhered to a part of the surface of the protection window correspondingto an unchanged area which is the individual area where the number ofthe pixels satisfying a third condition is equal to or greater than athreshold pixel number, the third condition being a condition that achange amount in a pixel value of each of the pixels included in theindividual area in a predetermined time period is equal to or smallerthan a threshold amount, when the camera image includes the unchangedarea, and not to perform the support control, when it has beendetermined that the protection window state has been the partial dirtstate.
 6. The monitor device according to claim 5, wherein, in the eventof the ECU determining that the protection window state has been thepartial dirt state, the ECU is configured to determine that the partialdirt state corresponding to the unchanged area ends, when the number ofthe pixels becomes equal to or greater than a third strength threshold,each of the pixels being included in the unchanged area and having anedge strength equal to or greater than a third strength threshold. 7.The monitor device according to claim 1, wherein the ECU is furtherconfigured to determine that the protection window state is the entiredirt state, when the own vehicle starts to be operated for running.